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Abstract 

The purpose of this research was to investigate the impact of immediate feedback through bug- 
in-ear eCoaching on early childhood special education pre-service teachers’ use of 
communication strategies using an activity-based intervention approach. Three early childhood 
special education pre-service teachers participated in this study. A multiple-probe, single-case 
design was used to determine the effects of immediate feedback through bug-in-ear eCoaching 
on teachers’ use of communication strategies. Results indicate that immediate feedback through 
bug-in-ear eCoaching enhanced pre-service teachers’ use of communication strategies within 
small-group activities. Implications for practice and future research are discussed. 

Keywords: bug-in-ear coaching, communication strategies, pre-service teachers, autism, 
personnel preparation 
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Pre-service Teacher Use of Communication Strategies upon Receiving Immediate Feedback 
Communication interventions for young children at risk for disabilities as well as those 
with disabilities is important as oral language is a critical skill that predicts reading 
comprehension (Lonigan & Shanahan, 2009) and school readiness (Rimm-Kaufman, Pianta, & 
Cox, 2000). Children who exhibit communication delays are at risk for developing problem 
behaviors and delays in literacy development (Horner, Carr, Strain, Todd, & Reed, 2002; 

Rayner, Foonnan, Perfetti, Pesetsky, & Seidenberg, 2001; Tager-Flusberg et ah, 2009). Children 
with autism frequently experience delays in communication and benefit from interventions 
targeting specific communication skills (Tager-Flusberg et ah, 2009). Therefore, it is important 
to identify interventions that produce promising communication outcomes for these children. 
Embedded Instruction for Young Children with Autism 

Embedded-communication interventions are those strategies occurring within typical 
classroom routines and activities that are designed to increase children’s functional 
communication skills. Numerous studies have demonstrated the effects of embedded- 
communication interventions on the communication development of children with autism 
(Rogers & Vismara, 2008; Tager-Flusberg et ah, 2009). Unfortunately, it is often a challenge for 
teachers in inclusive settings to use embedded interventions (Dinnebeil, Pretti-Frontczak, & 
Mclnerney, 2009; Jung, Gomez, Baird, & Keramidas, 2008). With strong evidence indicating the 
effectiveness of embedded interventions to support young children’s development (Ozen & 
Ergenekon, 2011; Rakap & Parlak-Rakap, 2011; Sandall, Schwartz, & Joseph, 2001), it is critical 
to train pre-service early childhood teachers to use embedded interventions within typical 


classroom environments. 
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Embedded-communication interventions are most effective when the intervention begins 
early, includes social uses of language, and occurs with sufficient intensity to facilitate 
acquisition and generalization of new skills (Hancock & Kaiser, 2002). Importantly, teaching 
communication skills in children’s natural environments has resulted in increased vocabulary, 
maintenance, generalization, and spontaneous use of language for children with autism (Mancil, 
2009). One approach that is implemented in the natural environment, includes the social use of 
language, and allows for high intensity and duration of learning opportunities is activity-based 
intervention (ABI). 

ABI is a child-directed approach to teaching that embeds learning opportunities on 
children's goals and objectives during routines, as well as planned and child-initiated activities 
(Macy, 2007; Ozen & Ergenekon, 2011; Pretti-Frontczak & Bricker, 2004). Teachers use 
logically occurring antecedents and consequences to develop functional and generalizable skills 
(Ozen & Ergenekon, 2011; Pretti-Frontczak & Bricker, 2004). ABI is a well-known, evidence- 
based naturalistic teaching strategy used throughout the day with children having a wide range of 
special needs (McBride & Schwartz, 2003; Ozen & Ergenekon, 2011). Although ABI is an 
effective approach (Pretti-Frontczak, Barr, Macy, & Carter, 2003), it is used infrequently or with 
limited fidelity by classroom teachers (Horn & Banerjee, 2009; McBride & Schwartz, 2003). 
Various intervention strategies, including those specific to communication, can be embedded 
using ABI. Thus, ABI can be particularly beneficial for children with autism, as their 
communication needs can be addressed multiple times throughout the day within highly 
motivating activities (Woods, Wilcox, Friedman, & Murch, 2011). 
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Preparing Pre-Service Teachers to Support Young Children with Autism 

Given that evidence-based practices, including ABI, are often not being implemented by 
teachers in the classroom, a research-to-practice gap exists (McLeskey & Billingsley, 2008). 
Researchers recognize that learning is enhanced when pre-service teachers are provided multiple 
opportunities to practice skills (Allsopp, DeMarie, Alvarez -McHatton, & Doone, 2006). 
Therefore, it is important to allow pre-service teachers opportunities to practice newly acquired 
skills, such as the ABI approach, during field experiences as this can help teachers apply 
knowledge and develop proficiency in new skills (Ostrosky, Mouzourou, Danner, & Zaghlawan, 
2013) so that children with autism receive the highest quality instruction. 

Feedback 

Feedback is one characteristic of quality early childhood special education (ECSE) field 
experiences that is malleable (Macy, Squires, & Barton, 2009); university supervisors and others 
working with pre-service teachers can modify and adjust feedback to enhance pre-service teacher 
learning. From their literature review of effective methods for providing feedback, Scheeler, 
Ruhl, and McAffee (2004) concluded that corrective, immediate, positive, and specific feedback 
are promising practices that result in changes in teacher behavior. Research indicates that 
teachers are able to adjust their practices when provided immediate feedback (Hemmeter, 

Snyder, Kinder, & Artman, 2011; Hsieh, Hemmeter, McCollum, & Ostrosky, 2009; Scheeler et 
ah, 2004). Researchers have since challenged this finding by suggesting that even delayed 
performance feedback has the potential to change teacher behavior (Solomon, Klein, & Polityl, 
2012). Although there is evidence demonstrating positive change in both delayed and immediate 
performance feedback, a plethora of researchers have examined the immediacy of feedback 
using bug-in-ear (BIE) coaching (Lindell, 2001; Scheeler, Congdon, & Stansbery, 2010). For 
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example, Herold, Ramirez, and Newkirk’s (1971) research provided early support that immediate 
feedback given to teachers using BIE coaching was more effective than traditional methods of 
providing feedback at the conclusion of a supervision session. 

Other researchers found less compelling results. For example, Giebelhaus (1994) found 
that pre-service teachers receiving BIE coaching from cooperating teachers outperfonned a 
control group of pre-service teachers who received traditional feedback from cooperating 
teachers on only one of the 14 measured behaviors. Further, outcomes of Lindell’s (2001) 
experimental study with pre-service teachers either receiving BIE feedback, delayed feedback, or 
no feedback were similar, with teachers in the BIE group outperforming the other groups in only 
one of seven behaviors. Although findings from these two studies seem bleak, there were several 
implications from these and other early BIE studies (Thomson, Holmberg, Baer, Hodges, & 
Moore, 1978; Van der Mars, 1988) that have influenced the implementation and, thereby, 
effectiveness of BIE coaching for pre-service teachers. For instance, university supervisors using 
BIE coaching now use promising practices such as targeting a small number of discrete-teaching 
behaviors (Scheeler & Lee, 2002), fading BIE coaching (Scheeler, McAfee, Ruhl, & Lee, 2006), 
planning for generalization (Scheeler, Bruno, Grubb, & Seavey, 2009), and measuring the type 
and content of BIE feedback (Kahan, 2002). 

The improvements in implementation of BIE coaching have resulted in better outcomes 
for pre-service teachers. For example, the five pre-service special education teachers in Scheeler 
and colleagues’ (2009) study improved upon their complete use of three-term contingency trials 
to at least 90% when receiving BIE coaching. In another of Scheeler and colleagues’ (2006) 
studies, BIE coaching was used to improve pre-service teachers’ instructional practices, and as a 
result, some children in their classrooms also improved in their targeted outcomes. 
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Recently, BIE technology has been used to provide distance, electronic coaching 
(eCoaching) to pre-service teachers (Rock et ah, 2009; Rock et ah, 2012; Scheeler, McKinnon, & 
Stout, 2012). In their seminal study, Rock and colleagues (2009) examined the use of BIE 
eCoaching on graduate-level teachers’ change in teaching behaviors and classroom climate. 

Their online, wireless technology included an ultra-wide angle web camera, a Bluetooth 
Headset™, a Bluetooth USB Adapter, and Skype™ video-conferencing software. Rock and 
colleagues found that BIE eCoaching improved teachers’ use of high-access teaching behaviors 
and praise statements. Furthennore, student engagement increased from 73.8% to 92.7%. 
Findings from these studies have led researchers to suggest that BIE eCoaching can improve pre- 
service teachers’ instructional practices and children’s outcomes. 

Scheeler and colleagues (2012) also demonstrated that BIE eCoaching can be an effective 
and feasible means to provide pre-service teachers with immediate feedback on their teaching 
practices. Furthennore, Rock and colleagues (2014) provide evidence that the pre-service 
teachers’ and children’s outcomes were sustainable with the removal of BIE eCoaching. While 
these studies provide evidence that immediate feedback provided via BIE eCoaching can 
increase teachers' use of evidence-based practices, this advanced technology has not been applied 
to ECSE pre-service teachers teaching in an inclusive environment where children with autism 
are a part of typical classroom activities and routines. ECSE settings are unique as they are 
typically inclusive environments where teachers are charged with a challenging task: meeting the 
needs of diverse learners including those who are typically developing as well as those with 
developmental delays and disabilities through naturally occurring activities and routines. Further 
research is needed to identify the effect of feedback on ECSE pre-service teachers’ practices, 
specifically related to effective-teaching strategies (ABI and communication strategies), in 
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inclusive environments, with children from high-need populations (i.e., autism). This study 
extends the research on BIE eCoaching by using short 10-min feedback sessions targeting ECSE 
pre-service teachers in inclusive environments while teaching at least one young child with 
autism. Further, using a new form of technology, Swivl, allowed the pre-service teachers the 
flexibility to navigate the classroom environment as they typically would during teaching 
sessions, which may enhance the feasibility of BIE eCoaching in ECSE settings where teachers 
are often moving around the classroom as they follow a child’s lead. 

Research Questions 

Although immediate feedback using BIE technology has been found to be effective in 
changing educators’ instructional practices (Scheeler et ah, 2004), using immediate feedback 
through BIE eCoaching is limited and using BIE eCoaching to prepare future teachers in 
inclusive settings where a young child with autism is present is a new phenomenon. To our 
knowledge, short (10-min), immediate feedback sessions via BIE eCoaching have not been used 
with ECSE teachers to increase their use of embedded, evidence-based practices in environments 
that present challenges for teachers related to embedding optimal learning opportunities for 
children. Additionally, replication of a practice (i.e., BIE eCoaching) is important to demonstrate 
a practice is indeed effective (Dunst, Trivette, & Cutspec, 2002; Homer et al., 2005; Odom et al., 
2005). The purpose of this study was to examine the impact of 10-min, immediate feedback 
sessions delivered via BIE eCoaching on pre-service teachers' use of communication strategies 
within an ABI framework. This study addressed two research questions: 

(1) How does immediate feedback provided during field experiences via BIE eCoaching impact 
ECSE pre-service teachers' use of communication strategies within an ABI framework? 
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(2) How do pre-service teachers rate the effectiveness of immediate feedback via BIE eCoaching 
in increasing their use of communication strategies? 

Method 

Participants and Setting 

Participants included three female pre-service teachers (Jordan, Shay, Noelle; 
pseudonyms) enrolled in a 15 -week student teaching internship during their final semester in an 
undergraduate ECSE licensure program at a large university in the Mid-Atlantic region of the 
United States. All participants were Caucasian, in their early 20s, and had two years or less 
experience working with children with disabilities. The first author invited pre-service teachers 
enrolled in student teaching who were not involved in other research projects (n = 4) to 
participate in the study. One pre-service teacher declined participation resulting in three 
participants. 

Intervention sessions took place using BIE eCoaching (described below) to provide 
immediate feedback from a distance during the pre-service teachers' regular student teaching 
hours. The pre-service teachers were placed in separate inclusive public-preschool classrooms in 
three elementary schools in the district serving the university community. Each classroom 
included up to 16 typically-developing children and four children with disabilities between three 
and five years of age. Implementation of immediate feedback intervention sessions using BIE 
eCoaching occurred during small-group activities (e.g., an insect dominoes game, an activity 
measuring plants, playing in sand table with funnels and scoops) planned and carried out by the 
pre-service teachers as part of their internship. Approximately four children participated in each 
activity. While group participation in the activities changed (e.g., one child may have 
participated in some activities but not in others), at least one child with autism was included in 
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each activity. The children with autism were diverse. Some of the children used single words 
and other used two-word phrases. All of the children with autism engaged in activities, but 
presented behavioral challenges at various times throughout the study. The children with autism 
were always included in the activities and a part of interactions, but the pre-service teachers were 
provided feedback based on their interactions with all children participating in small-group 
activities as the goal of this research was to determine the impact of immediate feedback on the 
pre-service teachers’ use of the communication strategies. Pre-service teachers were instructed to 
use strategies with children who were identified with autism, children presenting developmental 
delays, and children who were typically developing. 

Measures and Materials 

Observational measure. An observational event-recording measure was used to record 
prompts provided by the first or second author and pre-service teachers' use of communication 
strategies during the first 10-min of each activity. During feedback, we coded the number of 
prompts delivered, the prompt type (choice making, in sight out of reach, sabotage, wait time; 
see Table 1), and whether the pre-service teacher responded to the prompt by using it (+) or not 
using it (-). We also coded the number of times the pre-service teacher used the strategies 
spontaneously (using a communication strategy without a prompt delivered). Feedback sessions 
were coded live by the first or second author as feedback was delivered and reviewed via video 
recordings immediately following each session to check for coding accuracy. 

Interobserver agreement. An undergraduate student and a graduate student naive to the 
study design coded 25% of randomly-selected video-recorded sessions across participants and 
phases. Interobserver agreement (10 A) was calculated using the total agreement method 
(Kennedy, 2005). Using this method to calculate IOA, the frequencies with which each behavior 
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occurred is tallied for the total session. Then, the number of agreements are divided by the total 
number of agreements plus disagreements and multiplied by 100%. IOA was 93.45% for number 
of prompts delivered by the coach, 82.89% for pre-service teachers’ correct prompted use of 
strategies, and 80.24% for pre-service teachers’ spontaneous use of strategies. 

Feedback effectiveness questionnaire. At the end of the study, pre-service teachers 
completed an online questionnaire developed by the third author to assess the overall 
effectiveness of the intervention. The purpose of the questionnaire was to answer research 
question two. The questionnaire included 12 Likert-scale items related to pre-service teachers’ 
perceptions of the effectiveness of BIE eCoaching. 

Research Design 

Our multiple-baseline, multiple-probe design study (Gast & Ledford, 2010; Homer & 
Baer, 1978; Homer et ah, 2005) was developed based upon What Works Clearinghouse’s 
Standards for Single-Case Design ( Kratochwill et ah, 2010), meeting standards as the 
independent variable was manipulated, inter-assessor agreement for at least 20% of data was 
collected over time, the effect was demonstrated three times , at least five data points were 
collected in baseline and intervention, and three probe points were collected just prior to 
introducing intervention for each case. Although only two data points were collected for 
maintenance and generalization, these phases are not evaluated when judging the overall quality 
of a single-case research design (Kratochwill et ah, 2010). A probe design was selected because 
it minimizes unnecessary baseline data collection when participants are unlikely to develop the 
targeted skill without intervention. Based on previous research (Ottley & Hanline, 2014) we 
believed pre-service teachers' use of communication strategies would be unlikely to change 


without intervention. 
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Five baseline data points were collected for all participants prior to the start of 
intervention with Jordan. When we began intervention with Jordan, we collected additional 
baseline data for Shay and probe data for Noelle. Intervention continued until a participant met 
the a priori criterion of using communication strategies more frequently than baseline in eight 
consecutive sessions. The a priori criterion was established to exceed What Works 
Clearinghouse’s Standards (minimum of five intervention data points; Kratochwill et ah, 2010). 
Eight sessions was the highest dosage of intervention the research team could provide within 
restrictions related to pre-service teachers’ timelines for completing their internship. After 
participants reached criterion, we collected data during two maintenance probes to analyze their 
use of the communication strategies without feedback, as well as during two generalization 
probes in a different type of activity (e.g., circle time, breakfast, choice time) to analyze whether 
the participants could take the skills learned in one activity and transfer them to another activity. 
Intervention 

The National Professional Development Center on Autism Spectrum Disorders has 
grouped ABI, Enhanced Milieu Training (EMT), and other interventions that share common 
naturalistic elements into the category of Naturalistic Intervention (Wong et ah, 2014). The 
feedback that the first and second authors provided was focused on four communication 
strategies within an ABI approach. The communication strategies are environmental components 
of EMT, a naturalistic early language intervention that has demonstrated effectiveness of 
increasing language complexity and functional communication of children with disabilities 
(Kaiser & Roberts, 2013). These four specific strategies were chosen as they can be embedded 
into ABI, and they demonstrate research evidence of effectiveness in enhancing communication 
outcomes for young children with autism (Coogle, Floyd, Hanline, & Kellner-Hiczewski, 2013). 
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Only four teaching behaviors were chosen as previous BIE researchers have demonstrated the 
effectiveness of focusing on a small number of targeted teacher skills (Scheeler et ah, 2004). The 
four communication strategies included choice making, in sight out of reach, sabotage, and wait 
time. Choice making is offering a choice between two or more materials or activities (i.e., would 
you like the brown or blue paper); in sight out of reach is providing materials in a child’s line of 
vision but in a place where they could not reach it (i.e., pre-service teacher holding bingo chips 
in her hand as opposed to placing them in front of the child); sabotage is creating a situation 
where help is needed (i.e., providing the paper but not the paint to complete a picture); and wait 
time is pausing before a predictable action (i.e., while singing a song, pausing to wait for a 
communication attempt). See Table 1 for definitions. 

The intervention consisted of a brief training and immediate-feedback sessions. The brief 
training was provided to pre-service teachers via a narrated PowerPoint presentation that 
provided infonnation related to using four communication strategies (see Table 1). Pre-service 
teachers were provided the narrated presentation using Dropbox once they completed baseline 
sessions and before they started feedback sessions. The immediate-feedback sessions lasted 10 
min and were delivered by the first or second author via BIE eCoaching directing the pre-service 
teachers to use four communication strategies while they were leading planned small-group 
activities. 

Several pieces of technology were used to facilitate the immediate feedback via BIE 
eCoaching while pre-service teachers were engaged in the act of teaching. The Swivl, a 
technology released in January 2014, was used in the pre-service teachers’ classrooms to hold an 
iPad with a built in web-camera and Skype™. The Swivl holds the iPad and allows it to move so 
that the camera function can capture the appropriate view by tracking the teacher’s movement 
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(using a remote in a lanyard). This pennitted the pre-service teachers to move about the 
classroom as they typically would without having to carry the iPad with them. All sessions were 
recorded using ECamm™ (for Mac) and Evaer™ (for PC) software which allowed for 
simultaneous video-recording of the pre-service teacher and children, and audio-recording of the 
first or second author providing feedback. This software was compatible with Skype™ and 
installed on the first and second authors’ computers to allow for recording of sessions. 

During intervention sessions, the first or second author delivered prompts from her 
university or home office to the pre-service teacher in the classroom using BIE eCoaching, 
which included a wireless Blue Tooth™ device (an ear piece worn by the pre-service teacher) 
and Skype™ with video installed on the first and second authors' computers and the iPad. The 
first or second author was able to see and hear the pre-service teachers and children, but the pre- 
service teachers only received audio-feedback via the Blue Tooth™ device. The video setting on 
Skype™ was turned off and the Blue Tooth™ was used to transmit audio-feedback so that the 
pre-service teachers could hear the feedback while minimizing distractions in the classroom. 

During feedback sessions, the first or second author would receive a call from a 
participant on Skype™ from her university or home office. The first or second author delivered a 
maximum of 9 prompts per session (mean = 6.60, range = 3-9). The first or second author’s goal 
was to provide one prompt per minute; however, as pre-service teachers began using strategies 
spontaneously and based on the type of activity, this was not always feasible (pre-service 
teachers were maximizing the strategies that could be used based on the type of activity they 
were implementing). During each prompt, the first or second author directed the participant to 
use one of the four communication strategies (e.g., "Give Max a choice of which color scissors 
he would like to use." [choice making]; "Put the glue out of reach and wait for him to request it." 
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[in sight out of reach]. At least two types of communication strategies were prompted in each 
session. The first or second author provided affirmative feedback for prompts used correctly 
[e.g., "Good job giving him a choice."] or corrective feedback for prompts used incorrectly [e.g., 
"Next time wait before giving him the crayons."]). These feedback prompts are consistent with 
Scheeler and colleagues’ (2004) literature review on effective methods to provide performance- 
based feedback, namely to provide specific prompts with immediate, positive, and corrective 
feedback opportunities. 

Fidelity of implementation. Fidelity of implementation (FOI) was coded for the same 
25% of videos that were coded for 10 A using a FOI checklist (see online supplementary 
materials) developed by the second author that examined: prompts delivered (i.e., number of 
prompts; type of prompts; specific nature of prompts); feedback given on participants' correct 
and incorrect use of intervention strategies; the number of different communication strategies 
prompted (i.e., at least two); and, adaptation of number of prompts provided based on 
participants' spontaneous use of the strategies. FOI was calculated by dividing the number of 
checklist items completed by the total number of items and multiplying the quotient by 100%. 
FOI for the current study was 93.22%. 

Procedure 

Prior to initiating the study, pennission was received to conduct the study from the 
university institutional review board and school district. The first author obtained informed 
consent from pre-service teacher participants and children enrolled in the classrooms in which 
the intervention would take place. Children whose parents did not provide consent were directed 
by the classroom teacher to engage in other activities during research sessions. 
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Baseline. No training was provided prior to baseline. Pre-service teachers had, however, 
received instruction on using communication strategies within two university courses prior to 
student teaching as part of their regular coursework. During baseline, pre-service teachers were 
asked to use a flip camera to record between five (Jordan) and 10 (Noelle), 10-min videos of 
themselves leading a small-group activity that included at least one child with autism. Videos 
were analyzed for number of spontaneous communication strategies used by the pre-service 
teacher. 

Intervention. After establishing a stable baseline, Jordan was asked to watch a narrated 
PowerPoint presentation that provided a description and examples of the four communication 
strategies. Then the first or second author delivered 10-min feedback sessions via BIE eCoaching 
two times per day for four days. The intervention was introduced to the remaining two 
participants following the multiple-baseline staggered-introduction schedule. The first and 
second author delivered a total of eight sessions to Jordan and Shay, but only seven sessions to 
Noelle due to difficulty establishing an Internet connection during one session. 

Maintenance and generalization. Participants recorded maintenance data when they 
reached the a priori criterion (i.e., eight sessions above baseline); given the technical difficulties 
noted above, Noelle began recording maintenance data after seven sessions. During maintenance, 
the first author asked participants to use a flip camera to record themselves leading two 10-min 
small-group activities across two days. After completing the two maintenance sessions, the first 
author asked participants to record themselves teaching for 10 min during a routine other than 
small-group activities (e.g., mealtime, circle time, choice time). 
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Data Analysis 

The third author graphed pre-service teachers' prompted and spontaneous use of 
communication strategies in each session. The authors analyzed data using visual inspection of 
graphs and calculation of effect sizes. The authors examined graphs for changes in level, trend, 
and variability within each phase (Kratochwill et ah, 2010). In addition, the authors examined 
data across phases for immediacy of the effect, overlap, and the consistency of the data across 
participants during baseline and intervention phases (Kratochwill et ah, 2010). The first and 
second authors calculated effect sizes using percentage of all non-overlapping data (PAND; 
Parker, Hagan-Burke, & Vannest, 2007; Parker, Vannest, & Davis, 2011) and robust 
improvement rate difference (IRD; Parker, Vannest, & Brown, 2009; Parker et ah, 2011). 

PAND was selected for its utility in detecting power in multiple-baseline designs (Parker 
et ah, 2007). PAND is the percentage of data remaining after all overlapping data points have 
been removed (Parker et ah, 2011). PAND was calculated by removing overlapping data points, 
calculating the ratio of remaining data to all data points, and subtracting the ratio from 1 to 
convert the effect size to a 0 to 100 scale (Parker et ah, 2011). PAND is correlated with other 
effect size metrics including Phi 2 (.90) and R 2 (.87; Parker et ah, 2007). 

Robust IRD was calculated as a second effect-size measure because it is equivalent to 
robust Phi, thus providing a common metric for future meta-analyses that might include this 
study (Parker et ah, 2011). Robust IRD is the difference in improvement rates between phases A 
and B. The first and second authors calculated Robust IRD by removing overlapping (i.e. 
improved) data points and then adding half of the total improved points to the proportions for 
each phase (Parker et ah, 2011). Robust IRD is the difference between these proportions. 
Distributing the improved data points across phases is what distinguishes robust IRD from 
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traditional IRD in which improved data points are accounted for only in the phase in which they 
occurred. Robust IRD is equal to Cramer's V and Cohen's Kappa (Parker et ah, 2011). 

For each participant, the first and second authors calculated effect sizes between baseline 
and intervention, baseline and maintenance, and baseline and generalization phases. For 
intervention, the first and second authors calculated effect sizes for both spontaneous and 
prompted use of communication strategies, whereas only spontaneous effect sizes were 
calculated for maintenance and generalization because prompting did not occur during these 
phases. The first and second authors calculated omnibus effect sizes across participants by 
averaging effect sizes across participants as recommended by Parker and colleagues (2009). 
Finally, Likert-scale item responses on the teachers’ questionnaires were summarized. 

Results 

Jordan 

Within the baseline phase, Jordan had a zero-celerating slope with one instance of 
communication strategies used across all five baseline sessions (see Figure 1). Once intervention 
began, there was an immediate effect on Jordan’s prompted use of communication strategies, and 
an accelerating and improving trend. During intervention, Jordan produced a correct response for 
89.47% of prompts. Variability ranged from 3-9 prompted uses per session with a mean (level) 
of 6.38. There were no overlapping data points between baseline and intervention. Similarly, the 
data revealed an immediate effect on Jordan’s spontaneous use of communication strategies and 
an accelerating and improving trend. Variability ranged from 0-30 uses per session with a mean 
of 1 1.88. There was one overlapping data point between baseline and intervention. In 
maintenance sessions, Jordan used 12 and 7 strategies spontaneously, with no overlapping data 
between baseline and maintenance conditions. In generalization sessions, Jordan used 0 and 23 
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strategies spontaneously with one overlapping data point between baseline and generalization. 
The effect sizes for Jordan ranged from 65% to 100% across intervention, maintenance, and 
generalization phases as measured by PAND and Robust IRD, indicating a moderate to strong 
effect of immediate feedback via BIE eCoaching on Jordan’s use of communication strategies 
(see Table 2; Parker et ah, 2007, 2009, 2011). 

Shay 

Within the baseline phase, Shay had a zero-celerating slope with one instance of 
communication strategies used across all eight baseline sessions. Once intervention began, there 
was an immediate effect on Shay’s prompted use of communication strategies, producing an 
accelerating and improving trend. Variability ranged from 3-7 prompted uses per session with a 
mean (level) of 5.13. There were no overlapping data points between baseline and intervention. 
Shay responded correctly to 80.39% of prompts. The data revealed a gradual effect on Shay’s 
spontaneous use of communication strategies and an accelerating and improving trend. 
Variability ranged from 0-17 uses per session with a mean of 7.75. There was one overlapping 
data point between baseline and intervention. In maintenance, Shay used 7 and 12 strategies 
spontaneously with no overlapping data between baseline and maintenance conditions. In 
generalization sessions, Shay used 4 and 5 strategies spontaneously with no overlapping data 
points between baseline and generalization. Shay had an effect size ranging from 63% to 100% 
as measured by PAND and Robust IRD, suggesting a moderate to strong effect throughout all 
phases (see Table 2). 

Noelle 

Within the baseline phase, Noelle had a zero-celerating slope with zero instances of 
communication strategies used across eight baseline and two probe sessions. Once intervention 
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began, there was an immediate effect on Noelle’s prompted use of communication strategies, 
producing an accelerating and improving trend. Variability ranged from 2-8 prompted uses per 
session with a mean (level) of 5.29. There were no overlapping data points between baseline and 
intervention. Noelle responded correctly to 84.09% of prompts. Similarly, the data revealed an 
immediate effect on Noelle’s spontaneous use of communication strategies and an increasing and 
improving trend. Variability ranged from 2-20 uses per session with a mean of 10.14. There were 
no overlapping data points between baseline and intervention. In maintenance, Noelle used 16 
and 24 strategies spontaneously with no overlapping data between baseline and maintenance 
conditions. In generalization, Noelle used 25 and 12 strategies spontaneously and had no 
overlapping data points between baseline and generalization. Noelle maintained an effect size of 
100% throughout all phases of the study signifying a strong effect (see Table 2). 

Results across Participants 

Data patterns were similar across participants, with minimal use of the strategies by any 
pre-service teacher prior to intervention. The amount of accurately implemented prompted 
strategies was also similar across participants at about six uses per 10-min session. This level of 
implementation is consistent with the amount of prompts that the first or second author was 
providing to participants. The change in spontaneous use of strategies was immediate for two 
participants and gradual for one, with positive accelerating trends for all participants. Variability 
was observed across participants for maintenance and generalization. Three demonstrations of 
an effect of the intervention (immediate feedback via BIE eCoaching) were observed, and 
because there are not specific guidelines for maintenance and generalization, this study meets 
What Works Clearinghouse Standards for Single Case Design Research (Kratochwill et ah, 
2010). Therefore, effect sizes were calculated to determine the magnitude of the intervention’s 
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effect for the participants. The omnibus test was used to calculate the size of effect across 
participants. The results indicated a strong effect of the intervention across participants 
evidenced by an effect size of 86% to 100% across all participants measured by PAND and 
Robust IRE) (see Table 2). 

Social Validity 

Social validity is a critical component of research as it measures the practical application 
of research (Homer et ah, 2005). The feedback effectiveness questionnaire was used to rate pre- 
service teachers’ perceptions of immediate feedback via BIE eCoaching on their use of 
communication strategies. Results suggest that pre-service teachers perceived immediate 
feedback via BIE eCoaching to be an effective way to change pre-service teacher practices and 
child outcomes (see Table 3). Specifically, pre-service teachers indicated immediate feedback 
through BIE eCoaching was a helpful intervention that improved their communication practices 
and helped improve the quantity and quality of children’s communication. Additionally, teachers 
were confident that they would continue using communication strategies in the future. 

Discussion 

Our results support prior BIE eCoaching findings that suggest feedback provided from a 
distance enhances teacher practices (Rock et ah, 2009; Rock et ah, 2012; Scheeler et al, 2012). 
This study adds to the literature base of BIE eCoaching by investigating a unique population of 
participants, ECSE pre-service teachers in inclusive environments serving young children with 
diverse abilities, including those with autism. Inclusive environments have become more 
common placements for children with disabilities, but have also been identified as environments 
that present challenges for teachers (Berry, 2010). This study addresses how 10-min immediate 
feedback sessions impact ECSE pre-service teachers’ use of embedded-communication strategies 
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within naturally occurring activities, which is an additional novelty to this research. Further, this 
study provides infonnation regarding immediate feedback, generalization across routines, unique 
participant and setting influences, and implementation science, all of which are novel to this line 
of inquiry. 

Effectiveness of the Intervention 

Our first research question addressed whether or not pre-service teachers would increase 
their use of communication strategies when provided immediate feedback using BIE eCoaching. 
As in other BIE eCoaching studies, the participants in our study increased their use of the 
targeted skill (Rock et ah, 2009; Rock et al, 2012; Scheeler et ah, 2012). Specifically, all 
participants increased their use of communication strategies. Additionally, data suggest that 
participants were able to maintain these strategies without feedback and generalize these 
strategies into some new routines. Our finding that teachers maintained their use of strategies 
aligns with results obtained by previous researchers (Rock et al., 2014; Scheeler et al., 2012) and 
our results that generalization may be possible extend the BIE eCoaching research. Additionally, 
it is important to consider the benefit of ABI as it is an effective teaching method in which 
teachers use naturally occurring routines and activities to embed learning goals tailored to the 
diverse needs of children within an inclusive ECSE teaching environment (McBride & Schwartz, 
2003; Ozen & Ergenekon, 2011). This method, however, is infrequently used by teachers 
(McBride & Schwartz, 2003). These findings demonstrate immediate feedback using BIE 
eCoaching may be an effective means in increasing pre-service teachers’ use of evidence-based 
communication strategies within an ABI framework in inclusive ECSE settings. An additional 
point of consideration related to the effectiveness of the intervention is the social validity data 
that suggest pre-service teachers found this intervention helpful in producing both teacher and 
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child change, which is consistent with previous BIE eCoaching findings (Rock et ah, 2009; Rock 
et ah, 2012; Scheeler et ah, 2012). 

Generalization across Routines 

Our study is unique to other BIE eCoaching studies as we collected generalization data. 
Data suggest pre-service teachers were able to generalize strategies into some new routines; 
however, variability exists related to the types of routines. Jordan, for example, used 0 
communication strategies during circle time; however, when she was working one-on-one with a 
child during choice time, she used over 20 communication strategies. Shay and Noelle, 
demonstrated generalized use of the strategies during their generalization sessions which took 
place during a mealtime routine, choice time, and gross motor play, all of which were child-led 
activities. These preliminary data indicate pre-service teachers were successful in generalizing 
strategies to individual or small-group child-led routines and activities; however, Jordan did not 
generalize communication strategies into a large-group, teacher-led activity (i.e., circle time). 
Because Jordan was the only pre-service teacher who engaged in a large-group, teacher-led 
activity within the generalization phase, it is difficult to determine if this was a result of the type 
of activity or pre-service teacher difference. Nevertheless, this finding suggests a need to conduct 
further research related to immediate feedback and generalization across routines to examine 
how effective strategies can be implemented in all types of activities. It is likely also necessary to 
specifically plan or program for generalization (Stokes & Baer, 1977) of strategies to detennine 
whether the strategies coached in one routine (e.g., small group activities) generalize into other 
routines (e.g., circle time) that may present more challenges for early childhood teachers. 
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Participant and Setting Influences 

Although all participants increased their use of communication strategies and had an 
accelerating trend within the intervention phases, there was a high level of variability in 
participants’ spontaneous use of strategies. This finding could have several explanations. Our 
anecdotal observations suggest this may be due to the participants’ comfort level, the nature of 
the instructional setting, and/or child characteristics. Pre-service teachers may need different 
levels of support. Jordan appeared to manage multiple contextual variables (receiving 
technology-enhanced feedback, leading an activity, managing technological issues, etc.) with 
more ease than Shay and Noelle. In addition, Jordan was in a highly-structured, adult-directed 
setting whereas Shay and Noelle were in less structured and more child-directed environments. 
Additionally, the children in each of the settings had different characteristics. For example, 
some children with autism were more communicative and more engaged than others, which 
anecdotally appeared to impact the number and types of communication strategies being used. 
Future research should continue to investigate the associations between contextual variables and 
targeted outcomes of BIE eCoaching. 

Although data suggest the intervention was effective for all participants, these findings 
indicate a need to differentiate supports between participants when providing immediate 
feedback using BIE eCoaching. This may mean providing an increased dosage of feedback for 
certain participants. Additionally, developing systematic fading procedures as participants 
progress through phases of the study may be effective in decreasing variability and increasing the 
effect of intervention across phases. Overall, as participants moved through each intervention 
session, they increased their use of the strategies. Most importantly, all participants significantly 
increased their use of communication strategies within all phases subsequent to baseline. These 



COMMUNICATION STRATEGIES 


25 


findings provide infonnation to consider such as dosage and fading procedures when developing 
future research related to tailoring intervention procedures to participants. 

Teacher Perceptions and Implementation Science 

Our second research question addressed how pre-service teachers rated the effectiveness 
of immediate feedback via BIE eCoaching. Similar to other BIE eCoaching research (Rock et ah, 
2009; Rock, 2012; Scheeler et ah, 2012), pre-service teachers had positive experiences with BIE 
eCoaching. Specifically, all pre-service teacher responses to the feedback effectiveness 
questionnaire suggested perceived positive change for both the pre-service teacher and child 
participants. This finding is an important consideration as the social validity of interventions is 
critical when considering mechanisms to bridge the research-to-practice gap. For example, 
Odom, Cox, and Brock (2013) describe how adopters of evidence-based interventions for 
students with autism consider the “appropriateness and feasibility of the practice or program” (p. 
236) when choosing which evidence-based practices to adopt and implement. 

Finally, this research adds to the implementation science literature base. That is, moving 
an effective practice (communication strategies within an ABI model in this case) from a known 
effective practice with little implementation in classrooms, to a practice taking place in the real 
world (Fixsen, Blase, Naoom, & Wallace, 2009; Odom et ah, 2013). Notably, Fixsen and 
colleagues (2009) identify coaching as an essential core component of implementing evidence- 
based practices in real-world settings. The data from this study provide preliminary evidence that 
providing immediate feedback to pre-service teachers using BIE eCoaching while working with 
children with diverse abilities, including those with autism, may provide an additional pathway 
to ameliorate the research-to-practice gap in inclusive early childhood classrooms and aid pre- 
service teachers in the application of evidence-based practices. 
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Implications for Practice and Future Research 

Future research to extend this line of study includes examining settings, dosage and 
fading procedures, and child outcomes. Examining the impact of immediate feedback across 
various routines and educational settings are topics for researchers to consider in learning more 
about pre-service teacher preparation and their use of evidence-based practices in inclusive 
settings. Further, differentiating the intensity with which prompts are delivered to participants 
may be an effective support option. In addition, intervention practices that use specific 
procedures, such as a fading phase, may support participants in maintaining a higher level of the 
dependent variable and in generalizing use of intervention strategies to new activities. For 
example, providing feedback across activities (e.g., small-group, circle time, mealtimes) to 
potentially support pre-service teachers in learning how to generalize these strategies across 
activities may have beneficial outcomes. An additional consideration is providing a fading phase 
to provide learning opportunities to children when support is lessened and removed. Examining 
child outcomes as a result of teacher use of evidence-based practices would provide insightful 
infonnation regarding the impact of immediate feedback beyond its impact on teacher behaviors 
(Ottley, Ferron, & Hanline, in press). 

Limitations 

Although this research demonstrates promising practices for increasing pre-service 
teachers' use of evidence-based communication strategies, it is important to identify the 
limitations of this research. First, this study was specific to three pre-service teachers with 
similar demographic characteristics and the findings should not be overgeneralized. The extent to 
which these findings will replicate with different participants in other settings warrants further 
investigation. Second, more data points within maintenance and generalization would have 
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increased our confidence in the findings regarding the maintenance and generalized use of the 
communication strategies. Third, providing the narrated presentation right before the start of 
intervention may be a limitation as it is difficult to detennine if the use of the strategies is the 
result of BIE eCoaching or the training plus BIE eCoaching. Finally, different procedures were 
used to collect baseline, maintenance, and generalization data than intervention data (flip camera 
versus iPad). Therefore, this may have created some limitations related to the pre-service 
teachers’ understanding of what was taking place in each phase. 

Conclusion 

Findings from this study suggest using 10-min immediate feedback sessions via BIE 
eCoaching has the potential to increase pre-service ECSE teachers’ use of one evidence-based 
practice: embedded-communication strategies through activity-based intervention. Findings 
indicate immediate feedback paired with BIE eCoaching may increase pre-service teachers’ 
application of embedded intervention practices such as communication strategies in inclusive 
environments with a growing population of children, those identified with autism. This research 
supports BIE eCoaching literature in two ways: participants’ increased use of targeted skills and 
their perceived effectiveness of BIE eCoaching. However, our research adds to the BIE 
eCoaching literature base by using new technology (i.e., swivl) and a complex single-case design 
(multiple probe) to investigate the impact of 10-min feedback sessions during naturally occurring 
activities on ECSE pre-service teachers’ use of communication strategies. An additional unique 
aspect of our research is that our results suggest immediate feedback via BIE eCoaching 
increased teacher’s use of communication strategies in an ECSE inclusive setting while 
supporting a growing population of children, those with autism. Unlike prior BIE eCoaching 
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studies, we provide both maintenance and generalization data, and identify considerations for the 
implementation science literature base. 

Special educators are learning much about evidence-based practices and enhancing child 
outcomes for children with autism. Still, it is critical to consider how to bridge the gap between 
research and practice. One way to ameliorate this gap is defining and providing high-quality pre- 
service teacher preparation. Instruction on evidence-based practices in the higher-education 
classroom alone is not enough to promote pre-service teachers’ use of these practices with 
children (Hemmeter et ah, 2011; Hsieh et ah, 2009; Rock et ah, 2013). Although the pre-service 
teachers in our study completed coursework on evidence-based practices, including 
communication strategies, in the semester prior to the study, their baseline data suggested they 
were not using these strategies in the classroom. This mirrors previous findings which suggest 
that although traditional didactic approaches to pre-service teacher training lack effectiveness, 
professional development activities associated with practice, interaction, collaboration, and 
linkage to outcomes are associated with increased teacher use of targeted skills (Hemmeter et ah, 
2011; Hsieh et ah, 2009; Rock et al., 2013). It is critical that university supervisors provide pre- 
service teachers practice opportunities using effective feedback methods. The findings from this 
study are unique as they demonstrate that using short (10-min) sessions with immediate feedback 
through BIE eCoaching has the potential to bridge the research-to-practice gap by increasing pre- 
service teachers' use of one evidence-based practice, specifically, embedded-communication 
strategies using an ABI approach within an ECSE inclusive setting while working with young 


children identified with autism. 
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Table 1 

Communication Strategies 



Strategy 

Definition 

Examples 

Exclusions 

Choice making 

A choice between at 

Would you like the 

Does not include 


least two items where 

any response would 

blue or yellow paint? 

basic, teacher 

directed questions 


be appropriate 

Which Lego would 

you like (among 

more than 2 

options)? 

that only have one 

correct response 

In sight out of 

Placing desired 

Placing pointer 

Does not include 

reach 

objects in a place 

needed for calendar 

objects that are up 


where the child can 

see it but cannot 

activity on high shelf 

high without 

purposeful 


reach it 

Holding onto 

container with 

blocks and asking 

child to request 

blocks rather than 

giving all blocks at 

placement 


once 
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Sabotage 


Wait time 


Creating a situation 

Providing materials 

Does not include 

where help is needed 

that require 

content related 

that provides a 

assistance to access 

items unless 

purposeful 

(e.g., game pieces in 

obviously used to 

communication 

a clear jar with tight 

create a 

opportunity 

lid) 

communication 

opportunity (e.g., 


Not providing all 

asking a child to 


necessary materials 

(e.g., paint without 

the paper or paint 

brush) 

count 100 beads) 

A 3 -sec or longer 

Before providing the 

Does not include 

pause before 

child their milk at 

when used with 

providing necessary 

breakfast wait for 

another strategy 

or desired materials 

child to 

(e.g., placing an 


communicate 

object in sight out 

of reach and 


Before getting a 

pausing for the 


favorite toy wait for 

child to request) 


child to 


communicate 
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Table 2 

PAND a and Robust IRD Within and Across Participants 


Phase 

Intervention 

(prompted) 

Intervention 

(spontaneous) 

Maintenance 

Generalization 

Participant 1 
PAND 

100% 

92% 

100% 

86% 

Robust IRD 

100% 

94% 

100% 

65% 

Participant 2 

PAND 

100% 

81% 

100% 

100% 

Robust IRD 

100% 

63% 

100% 

100% 

Participant 3 

PAND 

100% 

100% 

100% 

100% 

Robust IRD 

100% 

100% 

100% 

100% 

Omnibus 

PAND 

100% 

91% 

100% 

95% 

Robust IRD 

100% 

86% 

100% 

88% 


'PAND is Percentage of All Non-overlapping Data. 
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Table 3 

Feedback Effectiveness Questionnaire 



Participant 1 

Participant 2 

Participant 3 

Receiving 
immediate feedback 
via bug-in-ear was 
helpful in changing 
my communication 
practices with young 
children. 

Strongly Agree 

Agree 

Strongly Agree 

Receiving 
immediate feedback 
using bug-in-ear is 
something that I 
would recommend 
to other students in 
pre-service teacher 
training programs. 

Strongly Agree 

Agree 

Strongly Agree 

The Swivl 
technology was 
helpful in the receipt 
of bug-in-ear 
feedback. 

Agree 

Agree 

Strongly Agree 

Using the 
technology to 
receive feedback 
was manageable. 

Strongly Agree 

Neutral 

Agree 

I currently use wait 
time very well. 

Agree 

Agree 

Agree 

I currently use 
choice making very 
well. 

Agree 

Strongly Agree 

Strongly Agree 

I currently use 
sabotage very well. 

Agree 

Agree 

Agree 
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I currently use in 
sight, out of reach 
very well. 

Agree 

I am confident that I 
will continue to use 
wait time without 
receiving bug-in-ear 
feedback. 

Agree 

I am confident that I 
will continue to use 
choice making 
without receiving 
bug-in-ear feedback. 

Agree 

I am confident that I 
will continue to use 
sabotage without 
receiving bug-in-ear 
feedback. 

Agree 

I am confident that I 
will continue to use 
in sight, out of reach 
without receiving 
bug-in-ear feedback. 

Agree 

When I used the 
communication 
strategies, the 
children 

communicated more 
regularly. 

Strongly Agree 

When I used the 
communication 
strategies, the 
children 

communicated more 
effectively. 

Strongly Agree 


Agree 

Strongly Agree 

Agree 

Strongly Agree 

Strongly Agree 

Strongly Agree 

Agree 

Strongly Agree 

Strongly Agree 

Strongly Agree 

Strongly Agree 

Strongly Agree 

Strongly Agree 

Agree 
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Figure 1 
Results 



Sessions 


